Theoretical Analysis of the k-Means Algorithm - A Survey

Clustering is a basic process in data analysis. It aims to partition a set of objects into groups called clusters such that, ideally, objects in the same group are similar and objects in different groups are dissimilar to each other. There are many scenarios where such a partition is useful. It may, for example, be used to structure the data to allow efficient information retrieval, to reduce the data by replacing a cluster by one or more representatives or to extract the main ‘themes’ in the data. There are many surveys on clustering algorithms, including well-known classics [45, 48] and more recent ones [24, 47]. Notice that the title of [47] is Data clustering: 50 years beyond K-means in reference to the k-means algorithm, the probably most widely used clustering algorithm of all time. It was proposed in 1957 by Lloyd [58] (and independently in 1956 by Steinhaus [71]) and is the topic of this survey. The k-means algorithm solves the problem of clustering to minimize the sum of squared errors (SSE). In this problem, we are given a set of points P ⊂ Rd in a Euclidean space, and the goal is to find a set C ⊂ Rd of k points (not necessarily included in P ) such that the sum of the squared distances of the points in P to their nearest center in C is minimized. Thus, the objective function to be minimized is